In the field of remote sensing technology, mainly in the usage of making high-resolution maps of the surroundings, to be used in many control and navigation applications such as but not limited to the automotive and industrial environment, gaming applications, and mapping applications, it is known to use time-of-flight based sensing to determine the distance of objects from a sensor. Time-of-flight based techniques include the use of RF modulated sources, range gated imagers, or direct time-of-flight (DToF) imagers. For the use of RF modulated sources and range gated imagers, it is necessary to illuminate the entire scene of interest with a modulated or pulsed source. Direct time-of-flight systems, such as most LIDARs, mechanically scan the area of interest with a pulsed beam, the reflection of which is sensed with a pulse detector.
In order to be able to correlate an emitted RF modulated signal with the detected reflected signal, the emitted signal must meet a number of constraints. In practice, these constraints turn out to make the RF modulated systems highly impractical for use in vehicular systems: the attainable range of detection is very limited for signal intensities that are within conventional safety limits and within the power budget of regular vehicles.
A direct TOF (DToF) imager, as used in most LIDAR systems, comprises a powerful pulsed laser (operating in a nanosecond pulse regime), a mechanical scanning system to acquire from the 1D point measurement a 3D map, and a pulse detector. Systems of this type are presently available from vendors including Velodyne Lidar of Morgan Hill, Calif. The Velodyne HDL-64E, as an example of state-of-the-art systems, uses 64 high-power lasers and 64 detectors (avalanche diodes) in a mechanically rotating structure at 5 to 15 rotations per second. The optical power required by these DToF LIDAR systems is too high to be obtained with semiconductor lasers, whose power is in the range of five to six orders of magnitude lower. In addition, the use of mechanically rotating elements for scanning purposes limits the prospects for miniaturization, reliability, and cost reduction of this type of system.
United States Patent application publication no. 2015/0063387 in the name of Trilumina discloses a VCSEL delivering a total energy of 50 mW in a pulse having a pulse width of 20 ns. The commercially available Optek OPV310 VCSEL delivers a total energy of 60 mW in a pulse having a duration of 10 ns and it can be estimated by extrapolation to have a maximum optical output power of 100 mW. This value is only realized under very stringent operating conditions, meaning optimal duty cycle and short pulse width so as to avoid instability due to thermal problems. Both the Trilumina disclosure and the Optek system illustrate that continuous-wave VCSEL systems are reaching their physical limits with respect to optical peak power output, due to thermal constraints inherently linked to the VCSEL design. At these pulse energy levels, and using ns pulses as presently used in DToF applications, the mere number of photons that can be expected to be usefully reflected by an object at a distance of 200 m is so low that it defeats detection by means of conventional semiconductor sensors such as CMOS or CCD or SPAD array. Thus, increasing the VCSEL power outputs by 5 or 6 orders of magnitude, as would be required to extend the range of the known DToF systems, is physically impossible.
Even the use of avalanche diodes (AD or SPAD), which are theoretically sufficiently sensitive to capture the few returning photons, cannot be usefully deployed in the known LIDAR system architectures. A solid state implementation of an array of SPADs must be read out serially. A high number of SPADs is required to achieve the desired accuracy. The serial read-out constraints of the solid state implementation limits the bandwidth of the system turning it inappropriate for the desired accuracy. For accuracies such as that of the Velodyne system (0.02 m to 0.04 m, independent of distance), the required read-out data rate exceeds the practically achievable bandwidth in case of today's IC implementation. For operation at 120 m, a SPAD array of 500×500 pixels is required, which, in an IC-based implementation, must be read-out serially. For the same precision as the aforementioned Velodyne system, it would require 1000 pulses per millisecond and hence 1000 frames per millisecond, translating into a readout rate of 250 Gigapixels per second. This is believed to be technically unfeasible in the context of current SPAD IC technology.
The paper by Neil E. Newman et al., “High Peak Power VCSELs in Short Range LIDAR Applications”, Journal of Undergraduate Research in Physics, 2013, http://www.jurp.org/2013/12017EXR.pdf, describes a VCSEL-based LIDAR application. The paper states that the maximum output power of the described prototype system was not great enough to do wide-field LIDAR at a range greater than 0.75 m. With a relatively focused beam (0.02 m spot size at 1 m distance), the authors were able to range a target object at a distance of up to 1 m.
The above examples clearly indicate that the optical power emitted by present semiconductor lasers cannot meet the power requirements necessary for operations in the known LIDAR systems to be of practical use in automotive applications (e.g. for ranges up to 200 m).
U.S. Pat. No. 7,544,945 in the name of Avago Technologies General IP (Singapore) Pte. Ltd., discloses vehicle-based LIDAR systems and methods using multiple lasers to provide more compact and cost-effective LIDAR functionality. Each laser in an array of lasers can be sequentially activated so that a corresponding optical element mounted with respect to the array of lasers produces respective interrogation beams in substantially different directions. Light from these beams is reflected by objects in a vehicle's environment, and detected so as to provide information about the objects to vehicle operators and/or passengers. The patent provides a solid state projector in which the individual lasers are consecutively activated in order to replace the known mechanical scanning in the known DToF LIDAR systems.
A high-accuracy medium-range surround sensing system for vehicles that does not use time-of-flight detection, is known from international patent application publication WO 2015/004213 A1 in the name of the present applicant. In that publication, the localization of objects is based on the projection of pulsed radiation spots and the analysis of the displacement of detected spots with reference to predetermined reference spot positions. More in particular, the system of the cited publication uses triangulation. However, the accuracy that can be achieved correlates with the triangulation base, which limits the further miniaturization that can be achieved.
U.S. patent application publication no. US 2012/0038903 A1 discloses methods and systems for adaptively controlling the illumination of a scene. In particular, a scene is illuminated, and light reflected from the scene is detected. Information regarding levels of light intensity received by different pixels of a multiple pixel detector, corresponding to different areas within a scene, and/or information regarding a range to an area within a scene, is received. That information is then used as a feedback signal to control levels of illumination within the scene. More particularly, different areas of the scene can be provided with different levels of illumination in response to the feedback signal. U.S. 2012/0038903 A1 does not disclose that the picture elements are configured to generate exposure values by accumulating a first amount of electrical charge representative of a first amount of light reflected by the object during a first predetermined time window and a second electrical charge representative of a second amount of light reflected by the object during a second predetermined time window, the second predetermined time window occurring after the first predetermined time window.
European patent application publication no. EP 2 322 953 A1 discloses a distance image sensor capable of enlarging the distance measurement range without reducing the distance resolution. A radiation source provides first to fifth pulse trains which are irradiated to the object as radiation pulses in the first to fifth frames arranged in order on a time axis. In each of the frames, imaging times are prescribed at points of predetermined time from the start point of each frame, also the pulses are shifted respectively by shift amounts different from each other from the start point of the first to fifth frames. A pixel array generates element image signals each of which has distance information of an object in distance ranges different from each other using imaging windows A and B in each of five frames. A processing unit generates an image signal by combining the element image signals. Since five times-of-flight measurement are used, the width of the radiation pulse does not have to be increased to obtain distance information of the object in a wide distance range, and the distance resolution is not reduced. The solution presented by EP 2 322 953 A1 consists of measuring charges representative of the respective parts of the reflection of a single pulse received in two consecutive time windows. As soon as a single reflected pulse has been received in the time windows A and B, the charges are transferred to the corresponding floating semiconductor areas for conversion into a “pixel image signal”.
European patent application publication no. EP 2 290 402 A1 discloses a range image sensor which is provided on a semiconductor substrate with an imaging region composed of a plurality of two-dimensionally arranged units, thereby obtaining a range image on the basis of charge quantities output from the units. One of the units is provided with a charge generating region (region outside a transfer electrode) where charges are generated in response to incident light, at least two semiconductor regions which are arranged spatially apart to collect charges from the charge generating region, and a transfer electrode which is installed at each periphery of the semiconductor region, given a charge transfer signal different in phase, and surrounding the semiconductor region. EP 2 290 402 A1 is not intended to work with a pattern of spots of laser light. Moreover, the solution presented by EP 2 290 402 A1 does not disclose using a periodically repeated sequence of pulses.
The article by Shoji Kawahito et al., “A CMOS Time-of-Flight Range Image Sensor With Gates-on-Field-Oxide Structure”, IEEE Sensors Journal, Vol. 7, no. 12, p. 1578-1586, discloses a type of CMOS time-of-flight (TOS) range image sensor using single-layer gates on field oxide structure for photo conversion and charge transfer. This structure allows the realization of a dense TOF range imaging array with 15×15 μm2 pixels in a standard CMOS process. Only an additional process step to create an n-type buried layer which is necessary for high-speed charge transfer is added to the fabrication process. The sensor operates based on time-delay dependent modulation of photocharge induced by back reflected infrared light pulses from an active illumination light source. To reduce the influence of background light, a small duty cycle light pulse is used and charge draining structures are included in the pixel. The TOF sensor chip fabricated measures a range resolution of 2.35 cm at 30 frames per second an improvement to 0.74 cm at three frames per second with a pulsewidth of 100 ns.
United States patent application publication no. US 2007/158770 A1 to Shoji Kawahito, discloses a range-finding image sensor based upon measurement of reflection time of light with reduced fabrication processes compared to standard CMOS manufacturing procedures. An oxide film is formed on a silicon substrate, and two photo-gate electrodes for charge-transfer are provided on the oxide film. Floating diffusion layers are used to convert charges to electronic potential, a mechanism traditionally inherited from the legacy technology of Charged Coupled Devices (CCD). Extra transistors are provided for resetting and a diffusion layer to provide a given reset voltage.
It is a disadvantage of the pixel disclosed in US 2007/158770 A1 that it uses non-standard technology and that the pixel design does not allow the addition of additional wells without sacrificing active surface area of the pixel. This is suboptimal for usage in sensor systems with ultra-low power lasers requiring large operational range. The used process is not commonly available in standard CMOS processes, which reduces this concept's applicability and its ability to be produced at an affordable cost in large volumes.
The range of a sensor based on such a design is also limited at the near end by saturation of the pixels by the strong reflections of projected light.
The saturation of pixels when sensing short-range reflections, or highly reflective objects such as traffic signs, license plates, etc., is especially problematic when the pixels are used in sensors for automotive applications, as is the purpose of the pixel according to the present invention, because Advanced Driver Assistance Systems (ADAS) and self-driving cars require high accuracy at short range. Moreover, in this application domain, accuracy at longer ranges, the ability to operate in bright ambient light conditions, and the requirement of compactness (requiring the use of solid-state semiconductor components) must not be sacrificed for the requirement of short-range accuracy.
United States patent application publication no. U.S. 2013/0148102 A1 aims to address erroneous measurements caused by multiple reflections in the scene, which are due to the parallel illumination and acquisition for all the pixels in today's state-of-the-art time-of-flight (TOF) range cameras. U.S. 2013/0148102 A1 proposes to compensate for the multi-path fusing the results obtained by applying two spatially different illumination schemes, typically one to achieve highest possible lateral resolution and for the second one structuring the emitted light and by doing so lowering the lateral resolution but limiting the impact of multiple reflections. However, the system described in U.S. 2013/0148102 A1 is a continuous-mode time-of-flight based sensor with global illumination, and the problems addressed in that document are inherent to global illumination. It is a disadvantage of global illumination schemes that they cannot realize the performance requirements of a semiconductor LIDAR when operating in range gating mode.
There is a continuing need to obtain extreme miniaturization and/or longer-range in complex vehicular surround sensing applications, such as ADAS (autonomous driving assistance system) applications and autonomous driving applications, and this at a reasonable cost and in a compact, semiconductor-integrated form factor, using technology apt for mass production.